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Abstract 

Models of real-time systems using a single para- 
digm often turn out to be inadequate, whether the 
paradigm is based on states, rules, event sequences 
or logic. In this paper, a model-based approach to 
reasoning about real-time systems is presented in 
which a temporal interval logic called TIL is 
employed to define constraints on a new type of 
high-level automata. The combination, called “Hi- 
erarchical Multi-State (HMS) machines,” can be 
used to model formally a real-time system, a dy- 
namic set of requirements, the environment, heuris- 
tic knowledge about planning-related problem solv- 
ing, and the computational states of the reasoning 
mechanisms. In this framework, mathematical 
techniques have been developed for (1) proving the 
correctness of a representation, (2) planning of con- 
current tasks to achieve goals, and (3) scheduling of 
plans to satisfy complex temporal constraints. 
HMS machines allow reasoning about a real-time 
system from a model of how truth arises instead of 
merely depending on what is true in a system. 

1. Introduction 

Real-time systems are characterized by unpredict- 
ability of inputs and “hard deadline” requirements. 
In addition, since many real-time systems are uti- 
lized in life-critical situations, strict “safety proper- 
ties” are usually defined for them. A safety property 
is a state of affairs that must always remain true in a 
system. Instead of the usual discussion of “liveness 
properties,” it is useful to define other requirements 


of a real-time system in terms of a set of “condition- 
al goals” defined in terms of (condition, goal) pairs. 
A condition defines the state of affairs under which 
the associated goal must be pursued. We assume 
that deadlines may be associated with goals and that 
requirements are dynamic so that the pursuit of an 
active goal may have to be abandoned if certain oth- 
er conditions become true. Thus, at the specifica- 
tion stage, the main forms of reasoning about a real- 
time system consists of the verification that (1) safety 
properties are not violated and (2) conditional goals 
are achievable. For traditional systems which oper- 
ate deterministically or stochastically, this is essen- 
tially sufficient even though it can be a very compli- 
cated process. At the operational stage, two other 
forms of reasoning arise for “intelligent systems” 
which are not defined deterministically and require 
a search or other forms of analysis to instantiate a 
specific set of responses in a particular situation. 
First, off-line reasoning can be performed to deter- 
mine in advance a set of allowable actions to achieve 
goals. Secondly, on-line reasoning can be 
employed, where deadlines on the reasoning pro- 
cess itself may have been defined. A key problem in 
the specification and operation of complex real- 
time systems is the choice of a representational 
framework that can provide manageable ap- 
proaches to specification, verification, and instanti- 
ation of behavior. 

While numerous formal representational schemes 
have been proposed for systems in general and real- 
time systems in particular, most of these are based 
on one of the following paradigms: state-based 
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models, rules, event sequences or logic. TWo major 
examples of state-based models are automata and 
Petri nets. For real-time systems, traditional 
automata are inadequate for at least two important 
reasons: (1) explosion of the state space for non-tri- 
vial systems, and (2) absence of a natural mecha- 
nism for representing temporal constraints. Petri 
nets reduce the state space and can represent con- 
current activities adequately. However, in Petri nets 
numerous dummy states are usually necessary to 
maintain logical consistency and no clear separa- 
tion is made between precedence and causality [7]. 
In addition, even timed Petri nets have a limited lan- 
guage for representing temporal relationships 
among states and events [6]. Specification and veri- 
fication of complex real-time requirements are also 
difficult for rule-based systems and event se- 
quences. In particular, it is generally accepted that 
while rules are appropriate for defining prototypical 
behavior, they are inadequate for reasoning about 
novel situations. As far as pure logical formalisms 
are concerned, temporal logic provides a promising 
approach, except for two shortcomings. First, cer- 
tain simple regular properties cannot be expressed 
in temporal logic [10]. Secondly, in a pure logic- 
based language a system is represented merely in 
terms of what is true. This gives a limited under- 
standing of system behavior, since knowledge of 


how truth arises which is common to state models is 
not readily available. 

The purpose of this paper is to present a brief over- 
view of a comprehensive framework for specifying 
real-time systems and reasoning about them, called 
“Hierarchical Multi-State (HMS) machines,” that 
integrates high-level “multi-state” automata and 
fragments of a temporal interval logic called TIL 
([7], [4], [3], [5], [6]). As noted in Figure 1, an HMS 
machine can be used to define formally the dynamic 
behavior of a system, its requirements, a model of 
the environment, heuristic knowledge about plan- 
ning-related problem solving, and the state of the 
computational resources used in reasoning. Given 
such a specification, the system can be simulated, its 
correctness can be verified formally, and it can be 
used for both off-line and on-line reasoning to de- 
rive operational plans and schedules to respond to 
the dynamics of a real-time situation. 

Section 2 presents an outline of a simple form of 
HMS machines, with a brief discussion of the meth- 
od for representing requirements in terms of “policy 
HMS machines.” Section 3 presents an overview of 
the planning process, plan representation languages 
and a scheduling algorithm for plans. Section 4 
presents a brief set of conclusions and directions for 
future work. 


Environment Planning 



Safety Properties 

Figure I. Specification, Verification and Reasoning Framework for Real-Time Systems 
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2. Automata, Temporal Logic, Machines and 
Real-Time Systems 

An automaton consists of a set of “states” and a set 
of “transitions” that cause changes in states due to 
the occurrence of certain events such as arrival of 
inputs. This provides a very general architecture for 
defining the dynamics of a system, except that, as in- 
dicated in the Introduction, it is inadequate for 
specifying complex real-time systems. Hierarchical 
Multi-State (HMS) machines [4] are high-level 
“multi-state automata,” in which (1) multiple (hier- 
archical) states can be true at one moment, (2) mul- 
tiple transitions can fire simultaneously, and (3) a 
temporal interval logic, called TIL, is used to define 
constraints on transitions. This architecture allows 
the compact definition of the dynamics of complex 
real-time systems, in which interactions among 
states and hard deadlines can be defined formally. 
In addition, a “multi-level” combination of HMS 
machines [5] provides the capability for formally 
defining dynamic requirements , giving rise to a mo- 
del-based reasoning framework for real-time sys- 
tems. Because of limitations of space, only the non- 
hierarchical version of HMS machines will be con- 
sidered here. A formalization of hierarchies can be 
found in [6]. 

An HMS machine is a triple H = (S, r D , r N ), where 
S is a set of “states,” Td is a set of “deterministic” 


transitions, and r N is a set of “nondeterministic” 
transitions. Boolean states represent properties 
that may be true or false about a system. Non-bool- 
ean states can represent both properties of multiple 
entities in a system and properties of data objects. 
Deterministic transitions denote fixed causal inter- 
actions among states, while nondeterministic tran- 
sitions represent possible or permissible interactions. 
Nondeterminism, in fact, is the key to the specifica- 
tion of choice in model-based reasoning in the HMS 
framework. 

The constraints or “controls” on transitions in an 
HMS machine are defined in terms of the temporal 
interval logic TIL which is obtained by adding the 
following three operators to propositional logic: 

O(t): At relative time t 
[ti, ti]: Always between times ti and t 2 
<ti, t 2 > : Sometime between times ti and t 2 

The operators [ti, t 2 ] and <tj, t 2 >, which allow 
hard real-time constraints to be defined for HMS 
machines, are generalizations of the standard tem- 
poral logic operators □ and O, respectively. All 
times are relative, with the current moment denoted 
by 0. Figure 2 depicts a simple 2-level example of an 
HMS machine specification that defines both a 
nondeterministic “basic machine” Hi and a specifi- 
cation of requirements in terms of the “policy HMS 


H 2 : 

(Policy Machine) 



At 

Airport 


Hi 



Figure 2. A 2-Level HMS Machine Specification of System and Requirements 
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machine” In this figure, rectangular boxes rep- 
resent states, dark arrows are transitions, thin ar- 
rows denote TIL controls on transitions with the 
symbol © next to each temporal operator, and the 
partially double-dashed arrow in H 2 is a “policy 
transition” that defines intentionality. Asterisks de- 
note nondeterministic transitions so that in Hi the 
choice of all actions is not completely determined. 
We say that a transition is “enabled” if (1) its “pri- 
mary” states from which the transition emerges are 
true, and (2) its controls are true. Thus, starting at 
the left side in the machine Hi, from the state ‘At 
Office” one can go into state “In Car” or into state 
“In Helicopter” as long as the control state “Work 
Finished” is true. If the state “In Car” (“In Helicop- 
ter”) is true, then nondetermini stically the transi- 
tion “Start Driving to Airport” (“Start Flying to Air- 
port”) is fired. Nondeterminism is useful since this 
machine may be part of the specification of a much 
larger set of behaviors that could include going to 
many other destinations. The horizontal bar from 
which the transition “Start Driving to Airport” 
arises is an infinite resource which is always true. 
Thus, if this transition fires, both the states “In Car” 
and “Driving to Airport” would be true simulta- 
neously. We note that at the end of this path, if the 
state “Driving to Airport” has been true continu- 
ously from 45 minutes earlier to the current mo- 
ment, then a deterministic transition will take one to 
the state ‘At Airport.” 

The policy transition of machine H 2 in Figure 2 de- 
fines the goal of reaching the state “At Airport” 
when executing Hi, with the requirement that the 
state “Going on Trip” must be true in the beginning 
and the trip should not take more than t minutes. 
Thus, depending on the value of t, different “plans” 
for Hi can be derived to reach the goal state. If the 
execution of the plan takes more than t minutes, 
then the plan can essentially be abandoned. Addi- 
tional types of controls on policy transitions that are 
not shown in the figure can be used to define com- 
plex interactions of states and goals, including the 
capability of making a goal dependent on the plan- 
ning process itself. Thus, for example, an alternate 


goal can be specified if the plan generation process 
takes longer than a specified length of time. Heuris- 
tic knowledge about plans can be captured by inter- 
mediate policy machines that define midpoint 
states that must be achieved during the execution of 
a plan. More details about policy machines can be 
found in [5]. 

An important benefit of the formal specification of 
a real-time system is that it provides a framework 
for verification of correctness and consistency be- 
fore implementation. Following the procedure in 3, 
given an HMS machine and any safety property de- 
fined on its states, one can create a new “extended” 
state that will be true if and only if the safety proper- 
ty is violated. By a result of [8], such a state need 
only depend on the past history of the states of the 
machine, even though safety properties are usually 
defined in terms of future events. Two specific veri- 
fication methods can then be used to verify that the 
extended state corresponding to the safety property 
is not reachable. In the first method, correctness- 
preserving transformations [3] are applied to 
modify an HMS machine incrementally, without af- 
fecting its behavior, until the safety state is isolated. 
In the second method, a “model-checking” ap- 
proach [6] is used to demonstrate in finite time the 
correctness of infinite behavior. As in [2], this in- 
volves a branching simulation process that termi- 
nates paths when cycles are detected. A major ad- 
vantage of using HMS machines is that orders of 
magnitude reduction in the number of states can be 
obtained in many applications compared to tradi- 
tional automata models. 

3. Planning, Plan Formalisms and Schedul- 
ing of Plans for HMS Machines 

A “plan” in the HMS framework consists of a se- 
quence of sets of transitions to be executed in a non- 
deterministic machine [5]. Conditional goals are 
specified for an HMS machine in terms of policy 
transitions of a policy HMS machine such as H 2 in 
Figure 2. The “planning” process then consists of 
searching the space of eligible nondeterministic 
transitions in a basic machine such as Hi to derive a 
plan that causes the goal states of a policy transition 
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to be reached. The important points to note in this 
framework are that (1) goals can be defined formally 
in terms of histories of states that are being modi- 
fied dynamically, (2) circumstances such as inability 
to meet a deadline may cause a goal to be dropped 
from consideration, (3) the states of the computa- 
tional resources in which planning is being per- 
formed may be used as controls on the policy transi- 
tions that define goals, and (4) heuristic guidelines 
for deriving plans can be specified in terms of inter- 
mediate policy machines. 

Compilation of plans in advance to meet goals with 
hard deadlines has been proposed by number of au- 
thors (see, e.g., [9]). Various representation 
schemes for plans have also been proposed. For ex- 
ample, in [1] a Petri net model is used to define con- 
ditional actions that depend on facts that are true 
about the environment. The HMS machine frame- 
work offers a powerful capability to define complex 
concurrent plans that depend not only on the cur- 
rent states of the world but also on temporal histo- 
ries of states. For this purpose, we say that a ma- 
chine P is a “plan HMS machine” for a nondeter- 
ministic machine H, if some of the states of P corre- 
spond to the nondeterministic transitions of H and 
some other states are “dependent” states of the 
states of H. A dependent state is defined as a state 
for which (1) truth only depends on a logical combi- 
nation of the truth or falsehood of other states, and 
(2) there are certain restrictions on transitions 
emerging from it and entering it. At each moment of 
time, the “execution” of P on H then is obtained by 
(1) firing the transitions of P as in a standard HMS 
machine, (2) firing the deterministic transitions of 


H, and (3) firing those nondeterministic transitions 
of H that are enabled in H and for which a corre- 
sponding state in P is true. Thus, for example, the 
plan machine in Figure 3 describes how the non- 
deterministic transitions in the machine Hi should 
be executed. The states containing asterisks are de- 
pendent states which, in this case, are simply dupli- 
cates of corresponding states in Hi, assuming that 
the state “In a Hurry” is added to H^ The states 
denoted by dashed rectangles represent transitions 
in Hi. Thus, this machine indicates that in case the 
state “In a Hurry” is true, one should execute the 
transition “Climb into Helicopter” from the state 
“At Office” in Hi. On the other hand, if the state “In 
a Hurry” is false, the transition “Get in Car” should 
be executed. Also, when the state “In Car” becomes 
true in Hi, the transition “Start Driving to Airport” 
will be fired if its corresponding state in Figure 3 is 
true. The latter situation will be true if the state 
“Going on Trip” has been true sometime earlier. 

TWo simpler formalisms for defining HMS machine 
plans can be defined in terms of the plan languages 
PL# and PLj y which can also be considered as lan- 
guages for describing concurrent event sequences. 
Words in the language PLo simply consist of se- 
quences of (1) symbols from the set of transitions of 
the HMS machine, (2) lists of symbols, (3) words 
with integer exponents. An individual symbol a de- 
notes the firing of the corresponding transition in 

the machine. A list of the form (a, p 8) denotes 

the simultaneous firings of the transitions a, p, 

8 . A word of the form w n represents the n-fold rep- 
etition of firing of the transitions in w. Thus, the 
plan a (p, y) (Stj ) 11 denotes the execution of the fol- 


j Get in Car j 


At * 
Office 

1 A 

i 


In a Hurry * 


Going on TVip * 


Start Driving to Airport | 


Climb into Helicopter I 




i 1 

Start Flying to Airport I 


Figure 3. A Plan Machine for the HMS Machine H, in Figure 2. 
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lowing transitions in a machine: first fire a, then 
fire p and y simultaneously, then fire 8 followed by r\ 
n times. The language PLj extends PLo by the intro- 
duction of conditional operators and the means for 
defining alternative choices of actions. Plans in 
such languages, combined with an underlying HMS 
machine, provide the capability for both model- 
based reasoning from basic principles and the abil- 
ity to respond rapidly to dynamic requirements 
without the need for searching. 

Plan languages also offer the possibility of studying 
the scheduling of plans as distinct from the planning 
or plan generation process itself. For example, con- 
sider the plan 

“Get in Car” “Start Driving to Airport” 

‘Arrive at Airport in Car” 

in the plan language PLo for the machine Hi of Fig- 
ure 2. This plan simply lists the sequences of actions 
that must be performed, in which there is a key 
missing element: when should the actions be per- 
formed. Here, the only missing part is a delay of 45 
minutes that must occur between the transitions 
“Start Driving to Airport” and “Arrive at Airport in 
Car.” If such required delays are incorporated into 
a plan and it is verified for correctness, then the un- 
derlying machine can essentially be ignored during 
the execution. The important correctness criteria 
for plans are: (1) no transition is attempted that is 
not enabled, and (2) the plan will transform the ma- 
chine from a given initial set of states to the desired 
final set of goal states. 

In [5] a general approach for deriving schedules for 
plans was introduced that also provides a limited 
method of verifying the correctness of plans. In this 
scheme, given a potential plan p’, a “variable delay 
plan” p is generated in which between each pair of 
terms in p’ a parametric delay is introduced, 
where 4> denotes a wait or “no action.” Using sym- 
bolic execution techniques, then a solution for the 
exponents of the 4>’s can often be found that guaran- 
tees the correctness of the plan. In addition, in 


many cases, misordered plans can be corrected in 
the process of finding the delays. 

4. Conclusions and Future Work 

Hierarchical Multi-State (HMS) machines provide 
a framework for specification, verification and con- 
trol of complex real-time systems by integrating 
multi-state automata and temporal interval logic. 
The major benefits are: (1) significant reduction in 
state space, (2) convenient mechanisms for specify- 
ing both safety properties and conditional goals, in- 
cluding hard deadlines, (3) methods of verifying cor- 
rectness of specifications, and (4) model-based rea- 
soning approaches for planning and scheduling in 
dynamic environments. 

Three directions for future work have been defined: 
theory, applications and tools. Theoretical research 
goals include (1) the extension and formalization of 
the specification language, (2) investigation of more 
powerful methods for capturing requirements, (3) 
verification methods, (4) representation of uncer- 
tainty relating to both incomplete knowledge about 
the world and probabilistic outcome of events, (5) 
introduction of learning, and (6) efficient planning 
and scheduling algorithms. Various potential appli- 
cation areas for HMS machine have also been iden- 
tified. Currently, HMS machines are being applied 
to the specification of a fragment of a future Euro- 
pean command and control system. As far as tool 
development plans are concerned, work is continu- 
ing on the development of a prototype environment 
for specification of HMS machines, along with the 
capabilities for interactive simulation, limited 
forms of animation, and verification. 
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